The present invention relates generally to the field of input/output (I/O) device control in a computer system and, more particularly, to the ordering of read return data relative to write data passing through a bus bridge between two computer buses where the presence of write posting storage in the bus bridge would otherwise cause ordering violations.
In a computer system, it is often desirable that direct memory access (DMA) reads by an I/O device use some form of prefetching so that memory access latency can be masked. As a result, I/O buses can be used more efficiently. It is critical, however, for the prefetching to be coherent to ensure that prefetched data supplied to DMA devices reflect the latest memory writes by processors, other I/O devices, or bus bridges. The coherent prefetch buffer can be thought of as a cache as it participates in the system's coherency protocol along with any other devices, such as central processing units (CPUs). Inasmuch as the prefetch cache is coherent, the completion of DMA traffic to the prefetch cache is equivalent, in terms of ordering and coherency, to completion to memory.
Writes by a CPU to an I/O device, also known as programmed I/O (PIO), can occur directly in a connected manner or through a posted write first-in-first-out (FIFO) buffer. If PIO is done is done via a direct connection to the I/O device through an I/O bus, then the prefetch cache coherency is sufficient to ensure that DMA read data from a cache hit is correctly ordered relative to PIO writes. Conversely, if a posted write FIFO is used, PIO writes are delayed in the posted write FIFO before they are written to the I/O bus. This delay can result in a read-write ordering violation.
This vulnerability to ordering violations is best illustrated by way of example. Consider a CPU that performs two writes A and B in that order. Write A is a PIO write to a device on an I/O bus. Write B modifies a location in memory. Because write A will be posted in the posted write FIFO, there will be some delay before the write is carried forward to the I/O bus. Write B, however, becomes effective immediately and is therefore visible to any I/O device reading that memory location. Therefore, if the I/O device that was written to by write A should read the memory location written to by write B while write A is still in the posted write FIFO, the effect of write B (i.e., the change in the memory value) will be visible to the I/O device before the effect of write A. Thus, an ordering violation will be incurred.
Accordingly, what is sought is a producer-consumer ordering system in which DMA reads instigated by a consumer agent are ordered relative to PIO writes that are instigated by a producer agent. Moreover, this ordering should be guaranteed even if the writes are to a mixture of different targets on different buses.